Theorem 6 is stated in terms of Gaussian complexity. Ben-David (2014) has a full proof. M (α)null is the linear class following the depth-K neural network. The second term relies on the Lipschitz constant of DNN, which we bound with the following lemma. Similar results are given by Scaman and Virmaux (2018); Fazlyab et al. (2019).

Neural Information Processing Systems http://nips.cc/

This enables learning on new tasks using far less data than is required to learn them in isolation.

Gaussian complexities (i.e. with more refined data-dependent bounds as R1 mentions), is an interesting future direction.

It has become a common practice (Tan et al., 2018) to use a pre-trained network as the representation